Bump types-setuptools from 83.0.0.20260807 to 84.0.0.20260812 - #780

…l brokered behind it

Route our self-hosted observation-portal deployment through Keycloak's
identity brokering instead of maintaining it as a second, directly-
integrated OAuth2 provider. Collapses pyobs-auth to a single-issuer
client, removes the AUTH_PROVIDER switch, and plans a full cutover of
archive's OAuth2Backend/BearerAuthentication rather than keeping them
as a permanent second code path. Also corrects the "odin" naming and
clarifies observation-portal is self-operated, not an external LCO
dependency.

Released as github.com/pyobs/pyobs-auth (private), 2.0.0.dev1 on
PyPI. Notes where the implementation diverged from the original
itemization: USER_RESOLVER is a pluggable per-service callable rather
than one shared sub-lookup, and login views/urls were added to
actually complete the browser-facing PKCE flow.

Login and logout confirmed working end to end against the real
auth.monet.uni-goettingen.de pyobs realm. Notes two deviations from
the original itemization: no Keycloak entry in AUTHENTICATION_BACKENDS
(the redirect flow doesn't fit that shape) and the accepted trade-off
of cutting over before observation-portal is brokered behind Keycloak.
pyobs-auth is now public.

…y graph

Offload lmfit.minimize and the residual model evaluation to a worker
thread so the coefficient fit doesn't block the event loop.

Offload image.to_bytes() to a worker thread when writing the image to
the cache, matching the existing create_jpeg() pattern, so a ~5MB FITS
write doesn't block the event loop.

…an Night note

…or fix

_basecamera only published IExposure state on status transitions, so progress
and exposure_time_left stayed frozen at 0 for the whole exposure. Add a
background task that re-publishes them every second while EXPOSING, and clamp
exposure_time_left to >= 0.

…de-prioritize two-vhost theory

…op step 4

… unsafe, decide warn

* Bound FITS-header fetch so a dead peer can't stall the frame

add_requested_fits_headers() awaited every header future with no timeout, so
a peer that never answers its IQ (e.g. a laptop put to sleep without closing
its client) stalled frame finalization for the full ~120s XMPP IQ timeout.
Bound the collection to a single asyncio.wait() deadline and skip anything
still pending. New fits_header_timeout kwarg (default 15s) on FitsHeaderMixin.
Fixes #764.

* Fix crash on empty header futures and fits_header_timeout not reaching BaseCamera/BaseVideo

asyncio.wait() raised ValueError on an empty futures dict (any module with no
comm, or no peer implementing IFitsHeaderBefore/After), crashing every
exposure. BaseCamera and BaseVideo also passed an explicit keyword list to
ImageFitsHeaderMixin.__init__ instead of forwarding **kwargs, so a configured
fits_header_timeout was silently dropped for exactly the module (fli230,
BaseCamera) that motivated this PR.

Adds regression tests for both, plus a fits_header_timeout-reaches-mixin
guard for BaseSpectrograph (already correct via **kwargs, previously
untested).

PR #765 merged. Records the two bugs found and fixed during review
(empty-futures crash, fits_header_timeout not reaching BaseCamera/BaseVideo)
and the follow-up filed as issue #767 for the narrow RemoteError catch.

* Add raw-frame streaming endpoint to BaseVideo alongside MJPEG

- New /video.raw multipart endpoint: JSON FITS-keyed meta header + raw
  little-endian bytes, event-driven with latest-frame-wins backpressure.
- VideoCapabilities.video -> mjpeg, add raw (both str | None); collapse
  live_view into video_path, add raw_path.
- Split add_fits_headers() into add_local_fits_headers() (no VFS I/O)
  plus the persistent FRAMENUM step; guard the per-frame centre warning.
- Add Image.from_ndarray() reconstruction helper.
- Fix video_handler's hardcoded 1 fps interval; lower sleep_time default
  to 60s.

* Fix raw-handler activity liveness, capture DATE-OBS at acquisition, document sleep_time trade-off

Addresses review findings on #766:
- raw_handler now bounds its wait on a new frame with a timeout, re-touching
  _active_time even when no frame arrives, so a connected-but-idle raw client
  can no longer let the camera sleep out from under it.
- DATE-OBS is now captured in _set_image() at acquisition time instead of at
  send time in _raw_frame(), avoiding drift under frame coalescing or
  scheduling delay.
- Documents the sleep_time default's cost trade-off in a code comment, as the
  plan required.

PR #766 merged. Records the three bugs found and fixed during review
(raw_handler liveness, DATE-OBS send-time stamping, missing sleep_time
trade-off comment) and the follow-up filed as issue #769 for the
per-client redundant frame-build work.

* Move event delivery from PEP presence auto-subscribe to explicit pubsub

Events now publish/subscribe via the shared pubsub.<domain> service using
explicit XEP-0060 subscribe/unsubscribe, mirroring the mechanism already
used for module state, instead of relying on XEP-0163 PEP + presence
auto-subscribe. A module only receives event types it actually registered
a handler for; add_interest()/PEP are no longer used for events at all.

Revises specs/adrs/0012 and the accompanying plan: the original design
(change PEP node access_model) depended on unverified ejabberd node_pep
behavior. Reusing the existing state-node mechanism sidesteps that
entirely and needs no ejabberd config change on any site.

Validated against the live tests/xmpp/docker-compose.yml harness: new
tests/integration/test_xmpp_event_subscriptions.py (4 tests, including
confirming a non-subscribed peer never receives the wire message at all)
plus the full existing XMPP integration suite, 34/34 passing.

* Address review: fix local-event over-subscription, module-less publish crash

Two blocking bugs from PR review:
- _got_online iterated all of self._event_handlers, including local
  events (ModuleOpenedEvent/ModuleClosedEvent, registered by every real
  Module) and stale empty-handler-list keys left behind by
  unregister_event(). Both spawned permanent retry-subscribe loops
  against pubsub nodes that will never exist. Now filtered to non-local
  events with at least one handler.
- send_event() dereferenced self._module.name unconditionally, ahead of
  its own "if self._module is not None" guard -- a module-less XmppComm
  (GUI, admin tool, observer) publishing e.g. a LogEvent via the logging
  handler crashed with AttributeError. Falls back to the JID's own
  username when there's no module.

Plus two lower-severity fixes:
- _unregister_events only unsubscribed from currently-online peers;
  unsubscribe targets the pubsub service itself and doesn't depend on
  peer presence, so it now covers every tracked subscription for the
  event, online or not.
- Added tests/integration/test_xmpp_event_subscriptions.py coverage for
  all of the above, plus the previously-claimed-but-untested restart
  and ModuleOpened/Closed sanity cases (7 tests total, all passing
  against the live ejabberd harness).

Also: ADR 0012 marked accepted (was proposed while already merging with
the implementation), specs/plans/index.md entries brought in line with
actual status, and logevent-double-delivery-fix-discussion.md moved
into specs/plans/ per CLAUDE.md.

* Remove stale root-level duplicate of the discussion doc

logevent-double-delivery-fix-discussion.md was committed at the repo
root before it got moved to specs/plans/ -- the move added the specs
copy but never deleted the original, leaving two copies with nothing
referencing the root one anymore.

* Gate event-node subscriptions on peer's advertised send role

Previously _got_online/_register_events subscribed to every online peer's
event node for every event type a module handles, regardless of whether
that peer actually publishes it -- e.g. a camera's BadWeatherEvent handler
would retry-subscribe to admin:BadWeatherEvent forever, spamming "still
failing to subscribe" on a node that will never exist.

_get_disco_info already tags each event in disco#info with role="send"/
"subscribe" (see _event_role); _get_interfaces now parses that into a new
_peer_sent_events cache, and the subscribe loops only fire for event
types the peer actually advertises sending.

* Add integration tests for send-role subscription gating

Covers the fix in 0690ec2: a peer that never advertises role="send" for
an event type must never get a subscription attempt (verified this fails
without the fix), and the flip side -- a peer that declares an event
handler-less (send-only) still gets subscribed to and delivers normally.

PR #761 merged. Corrects the stale test count (4 -> 9) and existing-suite
figure, and documents the send-role subscription gating fix added during
review.

* Make pydantic config models reject unknown keys (extra="forbid")

A task YAML misplaced guiding_config/acquisition_config inside
instrument_configs; pydantic silently dropped both keys instead of
erroring, and the task then failed can_run forever with no error
pointing at the config. Every pyobs BaseModel/PolymorphicBaseModel
now rejects unrecognized keys at load time instead of dropping them.

Also declares the LCO portal models' previously-undeclared fields
(state/submitter on LcoSchedulableRequest, instrument_name/
guide_camera_name/summary on LcoConfiguration, and eight fields on
LcoObservation) rather than opting them out with extra="ignore" -
the portal is self-hosted, so a schema mismatch should fail loudly
at upgrade time, not get silently absorbed forever.

Fixes surfaced along the way:
- create_object/get_object now route comm/timezone/vfs/observer
  through pydantic's validation context for pydantic models instead
  of passing them as constructor kwargs, which extra="forbid" would
  otherwise reject.
- Task is now a PolymorphicBaseModel so it pops its own `class` key.
- Merit.create() left a stale `type` key in the config dict after
  deriving `class` from it; only surfaced once the LCO fixture-setup
  error that had been masking it was fixed.

* Address review feedback on pydantic extra=forbid PR

- Constraint.create() had the same stale-type bug as Merit.create():
  it derived config["class"] from config["type"] but never removed
  type, so extra="forbid" now rejects it (Constraint is a
  PolymorphicBaseModel). Fixed the same way, plus a regression test
  reproducing the type-shorthand config path used by e.g.
  OnDemandScheduler(constraints=[{"type": "Airmass", ...}]), which no
  existing test covered (they all pass Constraint instances).
- Removed Merit.create()'s dead "dotted type" branch: it never set
  class in the first place, so it was already broken before this PR;
  half-fixing it by conditionally deleting type left it half-broken
  in a different way. Added the equivalent regression test for
  Merit.create()'s type-shorthand path.
- create_object()'s pydantic branch now passes by_alias=True to
  model_validate (matching Merit.create/Constraint.create's existing
  convention) and asserts against positional args, which model_validate
  can't accept.

* Fix LcoObservation required fields: two endpoints, two shapes

Checked the review's "verify required fields against a live portal"
concern against the actual portal source (LCO's Django app, our
self-hosted deployment) instead of guessing from fixtures.

Portal.observations() and Portal.download_schedule() hit different
endpoints with different response shapes:
- download_schedule() -> GET /api/observations/, routed through
  ListAsDictMixin.list() -> Observation.as_dict() with no args ->
  no_request=False -> observation_as_dict() sets all 8 fields
  unconditionally. This is the shape the test fixtures modeled.
- observations() -> GET /api/requests/{id}/observations/, a custom
  action that explicitly calls o.as_dict(no_request=True) -> those
  8 fields are omitted entirely, and `request` stays a bare FK id
  instead of an expanded object.

Made created/modified/ipp_value/name/observation_type/proposal/
request_group_id/submitter optional on LcoObservation. Added a
regression test against the actual no_request=True shape; verified
it fails with exactly the 8 missing-field errors against the
pre-fix code before restoring the fix.

* Harden create_object kwarg guards, dedupe merit/constraint type-shorthand logic

create_object's pydantic branch now raises instead of silently letting
kwargs clobber colliding cfg keys, and the positional-args guard is a
real raise instead of an assert (stripped under python -O). Also
extracts the duplicated type->class shorthand resolution out of
Merit.create/Constraint.create into a shared helper.

* Close sibling-repo question on Task.model_dump() class-key leak into pyobs-robotic-backend

Verified against pyobs-robotic-backend source: task.id is never None on
the reachable BackendObservationArchive+BackendTaskArchive path, and the
hypothetical mixed-backend case was already broken pre-PR (PK-only
ForeignKey field) independent of the class key, which the backend
strips defensively anyway.

* Pass by_alias=True on polymorphic-dispatch deserialization

retrieve_class_on_deserialization was the one model_validate call left
without by_alias=True, unlike Constraint.create/Merit.create/
create_object. Dormant today (no constraint/merit uses aliased
fields), but a polymorphic model with an aliased field would silently
fail validation on load without this.

Records the merge commit and the final round of review fixes
(create_object kwarg guards, deduped type-shorthand helper,
by_alias on polymorphic dispatch). Closes #755.

* Backfill CHANGELOG.rst for v2.0.0.dev53 through dev78

pyproject.toml had drifted to dev77 (soon dev78) while the changelog's
top entry was still an undated dev53 "unreleased" heading, leaving 22
released versions (dev54-dev77) with no entries at all. Split the
bundled unreleased section into its real dev53/dev54 releases, dated
each entry from pyproject.toml's version-bump commit history, and
wrote the missing dev55-dev77 entries from their commit diffs plus
specs/plans/steering docs. Added a new dev78 (unreleased) entry for
the two commits already shipped on top of dev77.

* Catch up whatsnew-2.0.rst for dev53-dev78

The page hadn't been touched since the commit right before dev53, so it
was missing every user-facing/breaking change since: the IVideo raw-frame
endpoint and video->mjpeg rename, pyobs.utils.pipeline.Night->Reduction,
HttpFileCache's Basic Auth->token auth switch, ImageProcessor.on_error,
OBSNUM, and BaseCamera's exposure-progress fix. Left out internal-only
perf/reliability fixes (event-loop offloads, IERS priming, log-level
tweaks) as not upgrade-relevant.

…plan and add anchor/alias tests

pydantic-extra-validation was merged (e398117, #762, closes #755) but still
filed as draft/not-finished in the index. Also revise object-kwarg-validation's
Decision: fix the comm_cfg anchor-holder leak at its source in pre_process_yaml
instead of allowlisting it in Object.__init__, since reload_anchors() already
identifies the leaking key by name. Add regression coverage for the
include/anchor mechanism in tests/utils/test_config.py, including an
xfail(strict=True) test documenting the comm_cfg leak itself.

* Fix comm_cfg anchor-holder key leaking into config dicts

pre_process_yaml's whole-file {include file} splices the entire included
file, including any key whose sole purpose is holding a YAML anchor for
<<: *anchor use elsewhere (e.g. comm.shared.yaml's comm_cfg: &comm). That
leaked key then reached Object.__init__'s **kwargs and was silently
dropped there -- masking real config typos, since the include leak looked
identical to a typo.

Drop anchor-holder keys from whole-file splices using the (keyword, anchor)
pairs reload_anchors() already extracts; keyed includes of the same key are
left untouched. Handle the resulting all-keys-stripped case (comm.shared.yaml's
only top-level key is the anchor holder) by dropping the splice placeholder
entirely instead of emitting an empty "{}" mapping, which broke YAML parsing
when followed by block-style content.

Verified against a real pyobs-monet config: comm_cfg no longer leaks, and the
comm: <<: *comm alias still resolves correctly.

See specs/plans/2026-08-09-object-kwarg-validation.md for the investigation
and remaining open items (environment/database wrapper keys, Object.__init__
enforcement level).

* Address review: scope empty-splice drop and anchor detection correctly

Two silent-data-loss bugs found in review of the comm_cfg anchor-leak fix:

1. The "drop an empty splice placeholder" special case wasn't scoped to
   whole-file includes, so a keyed include that legitimately selects an
   empty mapping (`{include file key}` where key's value is `{}`) silently
   became `null` instead of `{}`.

2. Anchor-holder detection reused reload_anchors(), which matches
   `keyword: &anchor` at any nesting depth via a plain (not line-anchored)
   regex. A top-level key could be incorrectly dropped from a whole-file
   include just because some unrelated *nested* key elsewhere in the file
   happened to share its name and carry an anchor.

Fixed by gating the empty-splice case on the same whole-file condition as
the anchor-drop itself, and by adding top_level_anchor_keywords() -- a
line-anchored regex restricted to unindented keys -- used only for the
drop decision. reload_anchors() itself is unchanged, since replace_aliases()
still needs to resolve anchors at any nesting depth.

Added regression tests for both. Full suite: 1490 passed, 25 skipped.
Re-verified against all 803 yaml files in pyobs-monet, pyobs-iagvt, and
pyobs-iag50: same result as before (2 pre-existing, unrelated errors; no
comm_cfg leaks in any consuming config).

comm_cfg fix (#773) and this pass together close out every confirmed
dead/misplaced/typo'd kwarg found by re-running the investigation as a
static check across pyobs-monet, pyobs-iagvt, pyobs-iag50, and
pyobs-polaris (815 real config files). environment/database, the last
open blocker on the Object.__init__ warn/raise decision, is confirmed
gone -- nothing found is blocking that decision anymore.

Most turned out to be ordinary installable PyPI packages, not
proprietary hardware SDKs. Found and fixed 4 real path/config bugs
(EAFFocuser and DummyTelescope moved/split, a dead name: key, a
47-line dead Zaber block) and 4 orphaned configs referencing
deliberately-retired or years-removed classes. Remaining unverified:
pyobs_gui.GUI (UI-only deps, not fleet-relevant), pyobs-pilar
(archived), pyobs-dashboard-utils (not cloned, skipped).

Two concurrent callers (e.g. an incoming video-stream request and an RPC
like set_exposure_time) could both observe self._active as unset and both
proceed to open/close the camera, double-connecting to the same GigE device
and tripping "Controller privilege required for streaming control" on
devices that only grant one controller. Serialize the check-and-set with a
lock.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

BaseVideo broadcasts NewImageEvent from _finish_image() but never declared
itself a sender via comm.register_event(NewImageEvent) (no handler), unlike
BaseCamera which does this in its open(). Without that declaration, a peer's
disco#info-driven subscription logic (xmppcomm.py's "skip event types this
peer doesn't actually publish") never subscribes to it, so send_event()
succeeds server-side but nothing ever receives it -- the GUI's video/FITS
grab silently never updates, with no errors on either side.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Implemented the straightforward raise-on-leftover-kwargs check; 59 of
1487 tests failed. Root cause: several classes (BaseTelescope and
others) fan the same **kwargs dict out to multiple sibling mixins
independently rather than a cooperative super() chain, so a key a
later mixin will legitimately claim looks unrecognized to
Object.__init__, which runs first via Module.__init__. Invalidates
the plan's original 'kwargs flow up until consumed' premise for
mixin-composed classes. Reverted the code change; documented two real
paths forward (fix the fan-out pattern, or check the full class-MRO
signature union instead of Object.__init__-local state).

The raise attempt reverted in object-kwarg-validation.md found a real
architectural blocker: several classes fan the same **kwargs dict out
to multiple sibling mixins independently instead of a cooperative
super() chain, so a key a later mixin will legitimately claim looks
unrecognized to Object.__init__, which runs first. Confirmed via AST
scan across every local pyobs-* repo: 28 production classes across 10
repos use this pattern, not just pyobs-core.

Chose fixing the pattern itself (cooperative super()) over an
alternative static MRO-signature-union check bolted onto
create_object -- the latter doesn't fix anything, it's a heuristic
that can be fooled by dynamic kwargs access (confirmed one real
instance: FiberCamera.rotation_correction_coefficients). Cooperative
super() makes Object.__init__'s own invariant true by construction.

New plan proposes a repo-by-repo rollout, safest first, each in its
own feature/* branch with a full test-suite gate, ending with the
actual raise once every repo threads kwargs cooperatively.

Bumps [types-setuptools](https://github.com/python/typeshed) from 83.0.0.20260807 to 84.0.0.20260812.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-setuptools
  dependency-version: 84.0.0.20260812
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>